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Highlights 

This report describes some of the key immediate and long-term outcomes achieved by the 
Appalachian Regional Commission (ARC)-Oak Ridge National Laboratory (ORNL) Summer 
Institute for Math/Science/Technology for student and teacher participants. This two-week 
summer program provides high school students and teachers from the Appalachian region the 
opportunity to work with mentor scientists from ORNL on inquiry-based, applied projects in 
science, math, and computer technology. The institute culminates with group, student and 
teacher, presentations about their projects. Other planned activities are designed to promote 
teamwork, expose students to college opportunities, and promote pride in the cultural richness 
and historical importance of the Appalachian region. 

The goals of the Summer Institute, in operation since 1990, are tol) encourage more high school 
students to continue their studies beyond high school, 2) encourage more students to pursue 
careers in the projected shortage areas of science, technology, engineering, and math (STEM), 
and 3) raise the level of math, science, and technology instruction in high schools throughout the 
region to facilitate the first two goals. 

The findings in this report are the result of an evaluation conducted by staff of the Academy for 
Educational Development, in which data were collected from eight cohorts of participants 
attending the Summer Institute between 1997 and 2004. The evaluation took place in 2005-06 
and is based on surveys from 89 students and 67 teachers, as well as interviews with a small 
number of participants. The evaluation has several limitations, including absence of a 
comparison group that would have made it possible to attribute outcomes solely to participation 
at the institute. Also, this evaluation was able to collect data only after participation in the 
institute, whereas with a carefully constructed pre-post comparison, the evaluation could have 
measured changes in student/teacher knowledge, attitudes, and intentions. 

Key findings for students and teachers follow. 

Students 

• Male and female students from each of the 13 states in the region participated in the 
Summer Institutes, and 31% of students came from counties designated as economically 
distressed by ARC. 

• Seventy-four percent of students said they felt more confident in their STEM abilities as a 
result of the institute. 

• Participation in the Summer Institute influenced 24% of students to take more science 
classes and 22% to take more math classes when they returned to high school. Somewhat 
more than half the students reported that their experience at ORNL reinforced prior 
decisions about the science and math courses they had already chosen to take (56% and 
52%, respectively). 

• All the student-survey respondents reported that, even prior to attending the Summer 
Institute, they planned to attend college. Even so, the students reported many ways that 
the institute reinforced their intentions to go to college and reduced some of the barriers. 



More than 50% of students reported that the program positively influenced their 
intentions to go to college. 

• Fully 96% of student participants who had graduated from high school at the time of the 
survey had continued their formal education. Of the 83 respondents who provided data, a 
great majority (79%) went to four-year institutions; 15 % went to community colleges or 
technical centers; 2% to military academies; and 3% did not continue their education 
beyond high school. 

• Of the 23 students who attended the institute in summer 1997 and 1998, all reported 
having attained higher education — 26% had some college, 39% had earned a bachelor’s 
degree, and 35% went on to graduate work. 

• After the Summer Institute, 36 students went on to earn a total of 46 degrees from higher 
education institutions. Overall, 54% of degrees earned were in STEM — 38% of associate 
degrees, 52% of bachelor degrees, and 86% of graduate degrees. 

• In total, 5 1 student-survey respondents indicated that they were currently pursuing post- 
high school education; 82% were pursuing degrees in STEM fields, including core 
subjects, health, and computer sciences. 

• Fifty-five percent of participants who were employed full-time reported that their jobs 
required proficiency in one of the STEM fields “to a great extent.” 

• The highlight of the experience named by students most often was interacting with 
people, especially their peers, from different regions of the country and meeting the 
ORNL scientists. 

• Many students reported that the Summer Institute experience helped them gain more 
mature perspectives socially and academically, an observation that was corroborated by 
teachers. According to many students, it was particularly significant to become aware that 
“there were others like me.” 

• Fifty-nine percent of students currently reside in the Appalachian region, and 53% 
anticipated residing in the region five years hence. There was no difference in intention to 
remain in the region among those who had majored in STEM or were currently majoring 
in STEM. 

Teachers 

• Teachers from each of the 13 states in the Appalachian region participated; 47% of 
teacher participants were from ARC-designated distressed counties. 

• Many teachers reported that they incorporated activities and approaches learned at the 
Summer Institute into their classrooms. When they returned to their classrooms, 77% 
drew on their experience at the institute for explanations and examples, 52% did so for 
classroom demonstrations, and 50% incorporated new knowledge in their lab 
experiments. 



• Eighty-six percent of teachers said that, following the institute, they encouraged students 
to continue their education in science, math or technology. 

• Sixty-nine percent reported talking to students in their classrooms about research 
applications in the STEM areas. 

• Seventy-five percent reported that they shared their experience and things they had 
learned with other teachers through informal conversations. Four teachers who gave 
formal presentations to colleagues estimated reaching a total of 95 teachers. 

• Fifty-seven percent of teachers said the institute encouraged them to seek more 
professional development in STEM. 

• Of the benefits of attending the Summer Institute named by teachers, they most often 
described the opportunity of meeting and networking with other teachers from the region. 

• Many interviewed teacher participants desired a more effective professional development 
component that would provide them with lesson plans and resources to better integrate 
their ORNL experience into their own classrooms. 

• Ninety-one percent of teachers currently reside in the Appalachian region, and 93% said 
they planned to reside in the region five years hence. 

Based on the findings, the evaluation report offers recommendations to help the institute fulfill 
its purpose more deliberately and effectively. The first recommendations listed below address the 
program overall, and then address specific student and teacher components of the program. 

Overall Program 

• Clarify recruitment criteria and process for students and teachers and make these criteria 
known to participants as well as the groups that recruit them. 

• Increase the number of youth from underrepresented groups in STEM. 

• Create connections with college-access programs in the region to expand the applicant 
pool and support student success over the long term. 

• Continue to conduct a mixed-method evaluation using a design that incorporates data 
collection before, and at two points following, participation in the Summer Institute. This 
will necessitate better student tracking. 

Student Components 

• Match student interests with projects or, if this is not possible, provide students with the 
opportunity to sample many projects. 

• Provide opportunities for students to meet or work with graduate students and young 
professionals. 



• Foster peer communication during and after the institute, both within and among different 
cohorts. 

• Create connections between the Summer Institute, sending school, and parents to support 
participants’ progress when they return home. 

Teacher Components 

• Give more attention to curricular issues by providing more structured time for teachers to 
discuss both specific curricular implications as well as ways to continue to promote 
interest in STEM education and careers among their students. 

• Foster networking among teachers from multiple years of the Summer Institute. 

• Make explicit expectations that teachers share their experience with other teachers in 
their schools and districts. 
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1. Introduction 



1.1 Background 

The Appalachian Regional Commission (ARC)-Oak Ridge National Laboratory (ORNL) 
Summer Institute for Math/Science/Technology is a partnership administered by the Oak Ridge 
Institute for Science and Education (ORISE), a program of the Oak Ridge Associated 
Universities. 

The Summer Institute was designed to address the declining availability of a scientifically 
literate workforce in the U.S., as well as to improve professional development for teachers in 
order to increase student achievement in these areas. Census data described below indicate that 
many counties within the Appalachian region lag behind the national average in college 
completion rates. ARC, seeking to promote economic development in the region, has funded this 
and other programs to increase the “intellectual capital” of the area. 

The institute’s three main goals are 1) encourage more high school students to continue their 
studies beyond high school, 2) encourage more students to pursue careers in the projected 
shortage areas of science, technology, engineering, and math (STEM), 1 and 3) raise the level of 
STEM instruction in high schools throughout the region to facilitate the first two goals. An 
unwritten but underlying goal of the program is to encourage students who obtain higher 
education, especially in STEM fields, and teachers who have achieved higher competencies in 
STEM, to remain in the Appalachian Region. To assess the results of its more than $2 million 
investment in the Summer Institute since its inception 16 years ago in 1990, ARC funded this 
evaluation to determine the extent to which these goals have been achieved. The evaluation 
sought to measure short- and longer-tenn outcomes aligned with the Summer Institute’s goals, 
including the following for students and teachers: 

Students 

• Participation in STEM courses while in high school 

• High school completion 

• College-going 

• Degrees pursued and earned in STEM and non-STEM fields 

• Employment in jobs requiring STEM proficiency 
Teachers 

• Incorporating activities and approaches learned at the Summer Institute into the 
classroom and sharing new knowledge with other teachers 

• Encouraging students to pursue higher education in STEM fields 



1 Throughout this report, for simplicity, we use the acronym STEM (science, technology, engineering and 
math) for all programs addressing at least three of these fields. 
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